On Greedy Algorithms for Decision Trees
نویسندگان
چکیده
In the general search problem we want to identify a specific element using a set of allowed tests. The general goal is to minimize the number of tests performed, although different measures are used to capture this goal. In this work we introduce a novel greedy approach that achieves the best known approximation ratios simultaneously for many different variations of this identification problem. In addition to this flexibility, our algorithm admits much shorter and simpler analyses than previous greedy strategies. As a second contribution, we investigate the potential of greedy algorithms for the more restricted problem of identifying elements of partially ordered sets by comparison with other elements. We prove that the latter problem is as hard to approximate as the general identification problem. As a positive result, we show that a natural greedy strategy achieves an approximation ratio of 2 for tree-like posets, improving upon the previously best known 14-approximation for this problem.
منابع مشابه
Decision Tree Construction using Greedy Algorithms and Dynamic Programming – Comparative Study
The paper presents comparative study for different heuristics used by greedy algorithms for constructing of decision trees. We consider the problem of exact learning for decision tables with discrete attributes. We made experiments with randomly generated decision tables contain attributes with three categories {0,1,2}. Complexity of decision trees is estimated relative to several cost function...
متن کاملAnytime Learning of Decision Trees
The majority of existing algorithms for learning decision trees are greedy—a tree is induced topdown, making locally optimal decisions at each node. In most cases, however, the constructed tree is not globally optimal. Even the few non-greedy learners cannot learn good trees when the concept is difficult. Furthermore, they require a fixed amount of time and are not able to generate a better tre...
متن کاملGlobal Tree Optimization: A Non-greedy Decision Tree Algorithm
A non-greedy approach for constructing globally optimal multivariate decision trees with xed structure is proposed. Previous greedy tree construction algorithms are locally optimal in that they optimize some splitting criterion at each decision node, typically one node at a time. In contrast, global tree optimization explicitly considers all decisions in the tree concurrently. An iterative line...
متن کاملEfficient Non-greedy Optimization of Decision Trees
Decision trees and randomized forests are widely used in computer vision and machine learning. Standard algorithms for decision tree induction optimize the split functions one node at a time according to some splitting criteria. This greedy procedure often leads to suboptimal trees. In this paper, we present an algorithm for optimizing the split functions at all levels of the tree jointly with ...
متن کاملDecision Tree Induction : How E ective is the Greedy Heuristic ?
Most existing decision tree systems use a greedy approach to induce trees | locally optimal splits are induced at every node of the tree. Although the greedy approach is suboptimal, it is believed to produce reasonably good trees. In the current work, we attempt to verify this belief. We quantify the goodness of greedy tree induction empirically, using the popular decision tree algorithms, C4.5...
متن کاملA less-greedy two-term Tsallis Entropy Information Metric approach for decision tree classification
The construction of efficient and effective decision trees remains a key topic in machine learning because of their simplicity and flexibility. A lot of heuristic algorithms have been proposed to construct nearoptimal decision trees. Most of them, however, are greedy algorithms that have the drawback of obtaining only local optimums. Besides, conventional split criteria they used, e.g. Shannon ...
متن کامل